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Learning  a  Procedure  from  Multimedia  Instructions: 

The  Effects  of  Film  and  Practice 

Abstract 

College  students  were  taught  to  build  a  model  helicopter  from  an  assembly 
kit.  Their  Instructions  consisted  of  a  narrated  film  (one  viewing  or  two), 
hands-on  practice  using  a  model  as  a  guide  (one  building  or  two),  or  a 
combination  (see  film  first,  build  second;  or  build  first,  see  film  second). 
Performance  on  assembly  from  memory  was  assessed  either  Immediately  or  after  a 
one-week  delay.  Both  structural  and  functional  measures  were  used.  (A  new 
structural  measure  Is  Introduced  here.)  Performance  was  best  Immediately  for 
groups  who  had  hands-on  practice,  either  twice,  or  In  conjunction  with  a  film. 
After  a  week,  the  group  who  practiced  first  and  saw  the  film  second  performed 
significantly  better  than  all  others.  A  theoretical  framework,  based  on 
multimedia  concept  formation.  Is  given  to  account  for  the  results.  In  order  for 
lasting  concepts  to  be  formed  In  memory,  a  precedence  Is  suggested:  motoric 
elements  should  be  put  In  first,  followed  by  visual,  followed  by  linguistic. 


Page  2 


Learning  a  Procedure  from  Multimedia  Instructions: 

The  Effects  of  Film  and  Practice 

In  this  study  we  tried  to  teach  people  to  perform  the  task  of  building  a 
fairly  complicated  object  from  a  kit  of  pieces.  The  object  to  be  built  was  a 
model  helicopter  made  from  the  Fischer  Technlk  50  assembly  kit.  We  varied  the 
type  of  Instruction  people  were  given,  that  Is,  the  stimulus  materials  that  went 
Into  the  Instruction.  Some  people  were  taught  by  viewing  a  narrated  film 
(either  one  viewing  or  two).  Others  were  taught  by  hands-on  practice  (building, 
either  once  or  twice,  using  a  physical  model  as  a  guide).  And  others  were 
taught  by  mixtures  of  film  and  hands-on  practice. 

The  purpose  of  the  experiment  was  to  measure  performance  as  a  function  of 
the  multimedia  Instructions  given.  By  multimedia  Instructions  we  mean 
Instructions  such  as  film,  written  text,  and  hands-on  practice.  Multimedia 
stimuli  can  be  roughly  divided  Into  visual,  verbal /auditory,  verbal/written,  and 
motoric  (actual  practice).  This  classification  Is  not  complete,  and  usually  the 
actual  stimulus  material  contains  more  than  one  element.  For  example,  during 
actual  practice.  If  the  person  Is  not  blindfolded,  a  visual  stimulus  Is  present. 

Other  researchers  have  studied  related  questions  using  multimedia 
Instructions  and  procedural  tasks.  One  popular  paradigm  used,  for  example,  by 
Margollus  and  Sheffield  (1963),  Cropper  (1968),  and  Robbins  (1983),  Is  to  vary 
the  amount  of  film  or  video  shown  before  a  person  Is  allowed  to  practice,  to  see 
If  there  Is  an  optimal  "work  unit”  size.  Stone  and  Glock  (1981)  gave  people 
written  Instructions,  with  and  without  pictures,  and  examined  different  kinds  of 
errors  people  made.  Stone,  Fortune,  and  Hutson  (Note  2)  allowed  subjects  to 
request  Information  from  different  modalities  to  see  If  there  was  a  preference. 
But  no  study  has  tried  to  answer  the  questions  posed  here,  and  no  study  has 
examined  groups  similar  to  the  ones  tested  here. 
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There  Is  always  a  question  In  procedural  tasks  about  how  to  test 
performance.  It  could  be  measured  using  many  different  media.  For  example,  a 
person,  after  trying  to  go  through  the  procedure  from  memory,  could  be  asked  to 
explain  what  he  or  she  did.  Or  the  person  could  be  asked  to  explain  what  he  or 
she  would  do.  Or  a  person  could  go  through  the  procedure  from  memory,  and.  his 
or  her  performance  could  be  assessed.  Or  the  person  could  look  at  an 
Incorrectly  built  object  and  spot  and  perhaps  correct  the  errors. 

The  performance  we  tested  was  building  from  memory,  either  Immediately 
after  receiving  the  Instructions,  or  after  a  one-week  delay.  The  reason  we 
chose  this  measure  Is  that  It  Is  a  direct  assessment  of  a  person's  ability  to 
execute  a  procedure  from  memory.  Further,  we  present  here  a  new,  objective, 
quantitative  method  for  scoring  the  finished  product  which  assesses  how  similar 
In  structure  an  object  built  from  memory  Is  to  the  original  object,  and  which  Is 
relatively  easy  (see  below).  Since  360  subjects  were  run,  ease  of  scoring  was 
an  Important  consideration. 

As  the  results  showed,  there  Is  a  large  variety  In  people's  performances. 
Using  a  conceptual  framework  (Baggett  and  Ehrenfeucht,  Note  1),  we  will  attempt 
(see  below)  to  explain  the  different  cognitive  processes  occurring  as  a  function 
of  the  Instructions  presented. 

Here  we  give  an  Intuitive  and  graphical  presentation  of  the  framework, 
without  giving  any  formal  definitions.  Mathematical  details  are  given  In 
Baggett  and  Ehrenfeucht,  Note  1. 

The  theoretical  framework  deals  with  concept  formation,  and  with  encoding, 
retaining,  and  using  Information  from  multimedia  stimuli.  As  mentioned  above, 
such  stimuli  Include  not  only  spoken  or  written  verbal  elements,  but  also, 
Importantly,  nonverbal  elements.  Including  still  pictures,  moving  pictures,  and 
actions. 

The  framework  uses  integrated  concepts.  That  Is,  each  concept  contains 
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elements  from  different  media,  which  correspond  to  words,  pictures,  actions, 
etc.  These  multimedia  concepts  have  a  hierarchical  structure.  That  Is, 
concepts  can  have  subconcepts,  which  can  have  subsubconcepts,  and  so  on. 

Within  the  framework  there  Is  a  single  conceptual  memory.  (Many  other 
researchers  also  assume  a  single  conceptual  memory,  for  example,  Anderson  and 
Bower,  1973;  Collins  and  Loftls,  197b;  and  Norman  and  Rumelhart,  1975.)  The 
memory  Is  connected  to  multiple  processors.  Processors  can  be  thought  of  as 
processing  visual  Input,  auditory  Input,  tactile  Input,  and  so  on.  Processors 
take  Input  signals  and  put  them  Into  memory,  forming  concepts. 

Concepts  are  not  Independent.  If  processor  A  builds  a  concept,  then 
processor  B  can  build  another  concept  that  Is  a  part  of  It.  This  gives  the 
hierarchical  structure  and  the  multimedia  aspect:  Visual,  auditory,  and  motoric 
Information,  for  example,  can  be  part  of  the  same  concept. 

Processing  of  concepts  Is  done  uniformly.  Independent  of  the  media  that 
went  Into  their  formation.  The  action  of  a  processor  Is  governed  by  a  concept 
already  In  memory.  A  concept  which  governs  what  a  processor  does  Is  called  a 
concept  which  Is  executed  by  the  processor. 

Besides  Interacting  with  memory,  processors  Interact  with  the  environment. 
For  example,  the  motoric  processor  can  cause  an  action  to  occur. 

Hypotheses  regarding  what  memory  Is,  how  concepts  are  located  and  linked  In 
memory,  and  how  concepts  are  formed  and  modified,  are  formulated  In  Baggett  and 
Ehrenfeucht,  Note  1. 

The  theoretical  approach  has  been  briefly  sketched.  The  concern  here  Is  to 
match  the  approach  with  experimental  data.  We  make  one  Important  point  before 
doing  so. 

We  assume  there  Is  a  distinction  between  having  a  slot  or  location  for 
Information  in  a  concept  and  filling  the  slot  with  specific  Information,  that 
Is,  giving  It  a  value.  Here  we  give  an  Informal  example  of  this  distinction. 
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Suppose  the  stimulus  1$,  "Chop  up  a  red  pepper."  The  Input  Is  verbal.  The 
concept  created  has  a  verbal  component  consisting  of  a  location  with  the  value, 
"Chop  up  a  red  pepper."  Its  visual  component  consists  of  a  location  for  color 
and  a  location  for  recognition  of  the  object.  (The  values  for  these  locations 
are  not  defined  by  the  Input.)  The  motoric  component  consists  of  a  location  for 
the  action  to  be  performed.  (Again,  the  value  for  the  motoric  location  Is  not 
defined  by  the  Input.) 

Let  us  now  change  the  stimulus  from  the  sentence,  "Chop  up  a  red  pepper," 

to  one  that  Is  visual.  Suppose  the  action  of  chopping  up  a  red  pepper  Is  shown 

via  a  (silent)  movie.  A  concept  similar  In  structure  to  the  one  above  Is 
created.  But  the  verbal  component  has  a  location  whose  value  Is  not  defined  by 
the  stimulus.  And  the  visual  component  has  locations  for  color  and  object 
recognition  whose  values  are  defined  by  the  stimulus.  The  motoric  component 
consists  of  location  for  the  action  to  be  performed,  but  Its  value  Is  not 
defined  by  the  Input. 

One  extra  mechanism  can  be  postulated  here,  namely,  the  notion  of  a  default 
value  for  a  location.  (See  Minsky,  1975.)  For  example,  with  the  verbal 
stimulus  "red,"  one  makes  a  concept  with  a  visual  slot  that  can  be  filled  with  a 

value,  and  the  value  for  the  visual  slot  can  be  filled  from  memory,  based  on  the 

typical  red  which  one  has  seen  previously.  If  such  a  mechanism  Is  operating, 
then  showing  red  to  a  person  later  on  can  be  unnecessary.  But  It  can  also  lead 
to  an  error:  the  exact  red  shown  later  may  not  be  encoded  (assigned  as  the 
value),  because  the  default  value  has  taken  precedence. 

The  questions  that  relate  experimental  data  to  the  framework  deal  with  the 
concepts  that  are  formed  depending  on  the  stimulus  materials  presented  during 
Instructions  (audiovisual  and  hands-on  practice).  We  assume  that  hands-on 
practice  causes  motoric,  visual,  and  verbal  locations  to  be  formed,  with  motoric 
and  visual  values  defined  by  the  stimulus.  Narrated  film  Instruction  causes 


motoric,  visual  and  verbal  locations  to  be  formed,  with  visual  and  verbal  values 
defined  by  the  stimulus. 

Suppose  practice  and  film  are  presented  In  some  order,  for  example, 
practice  first  or  film  first.  What  processing  goes  on?  We  assume  the  subject 
creates  one  concept  for  the  first  Instruction,  and  a  second  for  the  second 
Instruction.  Our  Interest  was  In  the  following  questions: 

1)  Can  a  person  transfer  values  from  one  concept  to  another  concept?  How  does 
transfer  occur  as  a  function  of  the  order  of  stimulus  presentation? 

2)  Can  a  person  create  one  concept  from  two  presentations?  Namely,  can  the 
person  Identify  one  concept  with  another,  so  that  they  become  one? 

The  difference  between  transferring  values  and  Integrating  concepts  can  be 
hard  to  distinguish.  We  assume  that,  when  tested  Immediately  after  receiving 
:  Instructions  (at  zero  delay),  a  subject  can  use  different  concepts,  simply 
because  they  have  been  formed  recently.  The  subject  can  create  a  composite 
concept  which  allows  him  or  her  to  solve  the  problem  at  test  time.  After  a 
delay,  we  expect  that  only  one  concept  Is  used  by  a  subject.  Therefore,  a  lack 
of  Integration  will  show  up  as  a  difference  between  performance  at  zero  delay 
and  performance  after  a  one-week  delay. 

In  this  study,  the  main  question  concerned  the  transfer  of  values  between 
motoric  and  visual  elements  obtained  during  practice,  and  visual  and  verbal 
elements  obtained  from  a  narrated  film.  We  measured  how  performance  depends  on 
the  mixture  and  Importantly,  the  order  of  the  stimulus  materials.  In  the 
discussion,  we  present  the  hypothetical  Interpretation  that  explains  the 
results,  and  an  Interpretation  of  when  concepts  were  formed  and  what  they 
contained.  We  also  can  specify  for  which  Instructional  sequences  the  transfer 
of  information  from  one  concept  to  another  Is  carried  out  efficiently. 


Method 


Subjects 

Three  hundred  sixty  students  in  Introductory  Psychology  at  the  University 
of  Colorado  participated  as  part  of  a  course  requirement.  The  students  were 
divided  Into  12  groups,  with  lb  males  and  15  females  per  group.  Subjects  were 
randomly  assigned  to  their  groups.  Six  of  the  groups  were  given  the  following 
Instructions,  with  subjects  tested  Immediately  afterwards:  group  on*:  first 
see  film,  then  build  from  model;  group  two:  first  build  from  model,  then  see 
film;  group  three,  see  film  twice;  group  four,  build  from  model  twice;  group 
5,  see  film  once;  group  6,  build  from  model  once.  The  other  six  groups  were 
given  the  same  Instructions,  with  subjects  tested  one  week  later.  The  test  for 
all  groups  was  to  build  the  model  helicopter  from  memory. 

Design 

The  experiment  was  a  6x2x2  between- subjects  design.  There  were  six  types 
of  Instruction  (1.  first  see  film,  then  build  from  model;  2.  first  build  from 
model,  then  see  film;  3.  see  film  twice;  4.  build  from  model  twice;  5.  see 
film  once;  and  6.  build  from  model  once),  and  two  delays  (0-  and  7-day). 
Further,  half  the  subjects  In  each  group  were  male  and  half  female. 

Stimulus  Materials 

The  assembly  kit,  Fischer  Technlk  50,  Is  similar  to  Lego.  Manufactured  in 

Germany,  the  kit  has  120  total  and  48  different  plastic,  metal,  and  rubber 

2  2 

pieces.  The  smallest  piece  Is  5mm  (.2  In  )  and  the  largest  Is  90  x  45  mm  (3.54 
In  x  1.77  In).  The  manufacturers  recommend  Its  use  by  children  as  young  as  six 
through  adults. 

The  object  to  be  built,  a  model  helicopter,  consists  of  64  totat  and  24 
different  pieces.  It  Is  shown  In  Figure  1. 


Insert  Figure  1  about  here 


The  Instructional  film,  a  1$  min  color  presentation,  was  shot  and  edited  by 
a  professional  film  maker,  James  Otis,  using  super  8  film.  It  was  narrated  by 
the  author.  The  narration  consisted  of  856  words.  The  film  presented  the 
helicopter  as  consisting  of  seven  subassemblies.  (In  a  different  study  these 
units  were  shown  to  be  the  same  as  those  of  the  majority  of  people  who  built  the 
helicopter  using  a  physical  model  as  a  guide.  They  can  thus  be  considered  to  be 
natural  units.  Details  of  the  method  for  determining  these  units  are  In 
Baggett,  In  press  a.)  Names  to  be  used  In  the  narration  for  the  24  different 
pieces  were  selected  using  a  method  described  by  Baggett  and  Ehrenfeucht  (1982). 
The  names  are  simple,  short,  easily  matched  with  their  physical  referents,  and 
fairly  well  recalled.  Names  to  be  used  In  the  narration  for  the  seven 
subassemblies  of  the  helicopter  (propeller  assembly,  seat,  upper  body,  lower 
body,  tall  assembly,  main  gear,  and  wheel  and  gear  assembly)  were  derived  In  a 
pilot  experiment.  In  the  experiment,  subjects  were  given  the  helicopter  broken 
Into  the  seven  subassemblies.  They  were  required  to  build  one  like  It,  with  the 
constraint  that  each  of  the  subassemblies  be  built  Individually.  After 
building,  each  person  named  each  subassembly.  Using  the  method  In  Baggett  and 
Ehrenfeucht  (1962),  the  most  commonly  generated  name  for  each  was  chosen  and 
used  In  the  film's  narration.  The  first  few  sentences  of  narration  are  given  In 
the  Appendix.  The  script  Is  available  from  the  author. 

Procedure 

Instructions  were  presented  to  Individual  subjects,  or  to  small  groups  of 
two  or  three.  Subjects  first  filled  out  questionnaires  Indicating  whether  they 
had  played  with  the  Fischer  Technlk  kit  or  with  Lego,  Erector  sets,  or  similar 
kits,  before.  They  rated  how  much  they  had  played  with  the  kits  on  the 
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following  scale:  0  (never);  1  (once  or  twice);  2  (fairly  frequently);  and  3 
(lots).  They  were  then  given  the  following  Instructions:  "Today  we  are  going 
to  give  you  some  Instructions  on  how  to  build  a  fairly  complicated  object  from 
this  kit  of  pieces.  After  the  Instructions  (and  at  the  appropriate  delay, 
either  zero  or  one  week)  you  will  be  asked  to  build  the  object  from  memory,  so 
try  to  learn  as  much  as  you  can  from  the  Instructions." 

They  were  then  told  that,  to  familiarize  themselves  with  the  kit's  48 
different  pieces,  they  would  first  do  a  matching  task.  Each  person  was  given  a 
box  containing  one  of  each  of  the  48  different  pieces  in  the  kit  and  four  sheets 
of  paper  with  48  names,  12  names  per  page.  They  were  instructed  to  place  each 
piece  by  its  correct  name.  They  were  told  that  the  task  was  not  a  test,  and 
that  they  could  ask  the  experimenter  at  any  time  what  the  name  of  a  piece  was. 
Subjects  finished  this  task  in  five  to  ten  minutes,  and  experimenters  then 
checked  and  corrected  their  errors,  pointing  out  the  corrections.  The  pieces 
and  their  names  were  then  removed. 

Subjects  were  then  told  what  their  instructions  would  consist  of  (one  or 
two  films,  building  from  a  model  once  or  twice,  building  from  a  model  once  and 
then  seeing  a  film,  or  seeing  a  film  and  then  building  from  a  model). 

Procedure  for  Showing  the  Film 

The  film  was  projected  on  a  38"  x  28.5"  (96cm  x  72cm)  screen  such  that  the 
visual  image  filled  the  screen.  Subjects  were  told  this  and  allowed  to  adjust 
their  chairs  before  the  film  began,  centering  them  before  the  screen  at  any 
distance  they  chose.  The  film  was  shown  In  a  completely  dark  room.  For  the 
groups  who  saw  the  film  twice,  the  break  between  showings  was  just  long  enough 
to  rewind  the  film  and  thread  It  again  Into  the  projector. 


.  i 
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Procedure  for  Building  from  £  Physical  Hodel 

Each  subject  was  run  by  an  Individual  experimenter  in  this  phase.  A 
physical  model  was  placed  before  each  subject.  The  subject  was  allowed  to  pick 
up  and  examine  the  model  at  any  time  and  could  disassemble  parts  of  It  If 
necessary.  In  building  his  or  her  copy  of  the  model,  the  subject  was  required 
to  ask  for  each  piece,  either  by  name  or  by  pointing.  At  the  subject's  side  was 
a  collection  of  each  of  the  48  different  pieces  In  the  kit  and  also  a  folder 
with  color  photos  and  names  of  each  of  the  pieces.  These  could  be  used  by  the 
subject  as  he  or  she  requested  each  piece.  The  experimenter  had  a  data  sheet  on 
which  each  of  the  pieces  of  the  helicopter  was  laid  out  and  numbered.  The 
experimenter  handed  the  subject  each  piece  requested,  and  recorded  the  order  of 
request  on  the  data  sheet,  as  the  subject  worked.  In  this  way,  the  experimenter 
could  check  that  the  subject  built  a  perfect  model.  (All  subjects  did  so.) 

For  the  groups  who  built  the  model  twice,  the  break  between  the  two  was 
just  long  enough  for  the  experimenter  to  disassemble  the  subject's  first  model 
and  place  the  pieces  back  in  the  kit. 

Subjects  In  the  Immediate  test  conditions  began  their  memory  trials  as  soon 
as  the  Instructions  were  completed.  Subjects  In  the  one-week  delay  groups 
returned  7  days  later  for  their  memory  trials. 

Procedure  for  Memory  Trial 

Each  subject  was  tested  by  an  Individual  experimenter  In  this  phase.  In 
building  from  memory,  subjects  were  required  to  ask  for  each  piece  they  wanted 
to  use,  either  by  name  or  by  pointing.  A  collection  of  the  48  different  pieces 
in  the  kit  and  a  folder  with  48  color  photos  and  names  were  at  the  subject's 
side  (as  above),  to  aid  In  this  task.  (All  48  different  pieces  were  available 
to  the  subject,  not  just  the  24  different  ones  occurring  In  the  correctly  built 
helicopter.)  Subjects  were  told  they  could  ask  for  as  many  different  pieces, 
end  as  many  pieces  of  one  kind,  as  they  wanted,  and  that  they  were  not  required 
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to  use  a  piece  once  they  had  asked  for  it.  They  were  also  told  that  they  could 
make  as  many  changes  as  they  wanted  while  they  built.  They  were  limited  to  one 
hour  for  the  memory  trial. 

Results 

For  convenience,  we  introduce  the  following  abbreviations  for  the  six 
groups:  FM  *  see  film  first,  build  from  model  second;  MF  *  build  from  model 
first,  see  film  second;  FF  *  see  film  twice;  MM  «  build  from  model  twice;  F  ■ 
see  film  once;  M  «  build  from  model  once.  The  FM  groups  tested  at  zero  delay 
and  7-day  delay  will  be  abbreviated  FM-0  and  FM-7  respectively,  and  other  zero- 
and  7-day  delay  groups  will  be  designated  similarly. 

In  building  the  helicopter  from  memory,  only  4.7%  (17  of  360  subjects) 
constructed  a  perfect  model  (two  from  FM-0;  six  from  MF-0;  three  from  FF-0; 
ana  six  from  MM-0).  The  degree  of  correctness  of  assembly  was  measured  by  two 
specific  criteria,  one  structural  and  the  other  functional. 

The  structural  criterion  measures  how  much  the  helicopter  built  from  memory 
differs  from  the  correctly  built  helicopter.  There  are  many  possible  structural 
measures.  For  example,  which  connections  were  made  incorrectly?  Were  some 
pieces  replaced  by  others,  with  possibly  the  same  functional  result? 

In  our  experiment,  we  used  a  new  quantitative  structural  measure  which  to 
our  knowledge  has  not  been  used  before:  number  of  correct  connections. 

Each  subject's  helicopter  built  from  memory  was  drawn  as  an  abstract  graph 
whose  nodes  represent  pieces  and  whose  links  represent  physical  connections. 

(As  an  example.  Figure  2  shows  the  abstract  graph  of  the  correctly  built 
helicopter  pictured  in  Figure  1.  Nodes  in  Figure  2  are  numbered  1  through  54, 
to  correspond  to  specific  pieces  in  the  helicopter.) 
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* 

Insert  Figure  2  about  here 


Each  subject's  graph  was  compared  to  the  graph  In  Figure  2,  and  correct 
connections  In  the  subject's  helicopter  were  counted.  For  example.  Is  piece  1 
connected  to  piece  2?  2  to  3?  3  to  4?  3  to  7?  (At  least  two  experimenters 
scored  every  helicopter.  When  there  was  disagreement  about  the  presence  of  a 
connection,  a  third  experimenter  scored  the  helicopter  Independently  and  broke 
the  tie.)  The  correctly  built  helicopter  contains  58  connections  (which  can  be 
seen  In  Figure  2).  Scores  on  the  helicopter  built  from  memory  could,  therefore, 
vary  from  zero  to  58. 

Table  1  presents  the  average  percentage  of  correct  connections  for  each  of 


Insert  Table  1  about  here 


the  12  groups  (and  the  percentage  functional,  to  be  explained  below).  The  6x2x2 
(Instructions  x  delay  x  sex)  between  groups  ANOVA  on  the  structural  scores 
yielded  a  significant  main  effect  of  Instructions,  F(5,336)  «  3.10,  p  <  .01 
(MS(error)  ■  125.4);  a  significant  main  effect  of  delay,  F ( 1 ,336 )  *  252.3,  p  < 
.01;  and  a  significant  main  effect  of  sex,  F( 1 ,336 )  *  55.1,  p  <  .01.  The 
Interaction  of  Instructions  x  delay  was  significant,  F(5,336)«  3.57,  p  <  .01. 

The  other  three  Interactions  (Instructions  x  sex,  delay  x  sex,  and  Instructions 
x  delay  x  sex)  were  not  statistically  significant  (F  <  1,  F  <  1,  and  F(5,336)  * 
1.28,  respectively). 

The  average  percentages  of  correct  connections  In  the  six  groups  (FM,  MF, 
FF,  MM,  F,  and  M),  collapsed  over  delay,  were  60.7,  66.4,  50.4,  63.3,  28.25,  and 
53.65  respectively.  The  average  structural  score  at  zero  delay  was  59.951; 
after  a  week.  It  was  37.62%.  (The  ratio  of  the  two,  a  measure  of  retention,  Is 
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53.8%.)  Overall,  males  scored  61.34%  and  females  46.23%. 

As  mentioned  above,  a  second  criterion  of  assembly  performance  was  an 
assessment  of  functionality  of  the  model  built  from  memory.  The  helicopter  Is 
operationally  defined  to  be  functional  if  its  blades  turn  when  it  Is  pushed 
along  a  surface.  Each  model  was  scored  as  functional  or  nonfunctional;  there 
was  no  partial  credit. 

The  scores  for  functionality  paralleled  the  structural  scores.  The 
percentages  functional  were  61.65,  65.0,  43.35,  55.0,  23.35,  and  48.35,  in  the 
six  groups,  ordered  as  above  and  collapsed  over  delay.  At  zero  delay,  55.02% 
were  functional;  after  a  week,  functionality  was  43.88%.  (These  numbers  give  a 
retention  rate  of  79.8%,  substantially  higher  than  the  retention  of  structure, 
53.8%.  That  function  Is  retained  better  than  structure  is  not 
surprising.)  Hales  built  65.57%  functional  helicopters;  33.33%  of  helicopters 
built  by  females  were  functional.  Statistical  analyses  were  not  performed  on 
the  functional  scores  since  their  pattern  was  very  similar  to  the  structural 
scores. 

One  Newman-Keuls  procedure  was  performed  on  the  structural  measure  to  test 
differences  between  pairs  of  means  (Winer,  1971,  p.  442)  at  zero  delay;  and  a 
separate  procedure  was  performed  to  test  differences  at  7-day  delay.  (In  each 
case,  data  from  males  and  females  were  combined.)  For  both  0-  and  7-day  delay, 
HS(error)  ■  145.1.  The  critical  value  for  a  .05-level  test  for  adjacent  means, 
q  95(2,348)  Sjg,  is  10.5%. 

Therefore,  at  zero  delay,  the  six  groups  line  up  as  follows  based  on  their 
structural  scores  (see  also  Table  1): 

MM  »  MF  »  FM  >  FF  •  M  >  F. 

Thus,  when  the  test  Immediately  follows  Instructions,  some  motoric  training  Is 
good:  either  practicing  twice  or  practicing  once  combined  with  seeing  a  film. 
(Order  of  practice  and  film  does  not  matter.)  Second  best  is  practicing  once  or 
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seeing  the  film  twice.  This  means  that,  at  least  when  the  test  Is  Immediate, 
audiovisual  Instruction  can  be  substituted  for  some  practice.  If  the  amount  of 
audiovisual  Instruction  Is  Increased.  We  also  note  the  Increase  In  performance 
from  the  F  to  the  FF  group  (36.8%  to  68.9%).  This  means  that  (Immediate) 
performance  resulting  from  audiovisual  Instruction  for  a  procedure  can  be 
significantly  Improved  when  the  Instruction  Is  shown  twice  rather  than  once.  We 
do  not  know  whether,  by  repeating  the  film  (giving  only  audiovisual  components 
to  a  concept),  we  could  ever  match  the  three  top  groups,  all  of  which  have  a 
motoric  component. 

After  a  week  delay  the  six  groups  line  up  statistically  (using  a 
Newman-Keuls  procedure  as  before)  as  follows  (once  again,  based  on  structural 
scores;  see  also  Table  1): 

MF  >  MM  «  FM  ■  M  «  FF  >  F. 

As  shown  In  Table  1,  all  groups  are  depressed  to  about  1/2  of  their 
zero-delay  scores.  But  there  Is  one  group  that  stands  out  above  the  rest:  MF. 
As  mentioned  above,  a  measure  of  retention  may  be  defined  as  the  score  at  7  days 
divided  by  the  score  at  zero  delay.  The  MF  group  retains  65%  of  what  was 
originally  encoded.  Retention  rates  for  the  other  five  groups  range  from  46%  to 


Discussion 

We  give  here  an  Interpretation  of  the  results  In  terms  of  the  theoretical 
framework,  l.e..  In  terms  of  the  concepts  formed.  Before  doing  so,  we  review 
two  points  mentioned  In  the  Introduction.  First,  depending  on  the  stimulus 
presented  In  Instructions,  the  concept  formed  can  have  some  values  for  locations 
missing.  For  example,  seeing  the  film  does  not  provide  motoric  values.  When 
motoric  values  are  not  provided  by  the  stimulus,  they  can  be  undefined  or 
defined  by  default.  (The  person  has  done  something  similar  before,  so  the 
.(motoric)  memory  of  the  similar  action  Is  used  as  a  value  in  the  new  concept.) 
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The  second  point  regards  whether.  In  the  different  groups,  one  or  two 
concepts  are  built.  Me  expect  that  In  the  groups  where  the  same  stimulus  Is 
repeated  (FF  or  MM) ,  one  single  concept  Is  formed,  and  the  repetition  simply 
reinforces  the  concept  formed  during  the  first  stimulus  presentation.  The 
repetition  will  yield  (1)  a  better  encoding  of  the  stimulus  presented;  and  (2) 
a  better  chance  that  the  missing  values  can  be  provided  by  default.  With  one 
presentation,  a  person  may  not  have  time  to  search  through  other  concepts  In 
long  term  memory  to  find  the  default  values  needed  to  fill  the  slots.  (We  think 
that  this  search  is  similar  to  the  reinstatement  search  hypothesized  by  Kintsch 
&  van  Dijk  (1978)  In  their  model  of  text  comprehension.) 

For  the  groups  who  receive  both  film  and  practice,  we  expect  that,  even  if 
two  separate  concepts  are  formed,  they  can  be  used  by  subjects  at  zero  delay  to 
perform  the  task,  simply  because  the  concepts  have  been  encoded  close  together 
temporally.  A  person  can  form  associations  between  the  two  at  zero  delay,  and 
use  them  both  to  solve  the  problem.  We  also  expect  that,  when  the  task  is 
performed  after  a  week  delay,  only  one  of  the  concepts  formed  will  be  used. 

That  Is,  the  chance  that  the  two  concepts  are  Integrated  Is  far  smaller.  As  we 
mentioned  In  the  Introduction,  we  hypothesize  that  the  difference  between  one 
Integrated  and  two  separate  concepts  can  be  detected  based  on  performance  with 
and  without  a  delay. 

We  also  hypothesize  as  follows  regarding  when  one  can  create  default  values 
for  locations  for  media  not  presented  In  the  stimulus  materials.  Before  default 
values  can  be  created,  locations  for  the  values  «re  required  to  be  present  in 
the  concept. 

We  first  line  up  the  media  as  follows:  motoric  first;  visual  second; 
verbal  third.  We  hypothesize  that  a  motoric  stimulus  creates  locations  for 
visual  (as  well  as  motoric)  values.  A  visual  stimulus  creates  locations  for 
both  motoric  and  verbal  (as  well  as  visual)  values.  Finally,  a  verbal  stimulus 
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creates  locations  for  visual  (as  well  as  verbal)  values.  That  Is,  locations  are 

i 

created  for  the  medium  presented,  and  for  the  media  one  step  on  either  side  of 
the  one  presented,  as  they  are  lined  up  above.  This  hypothesis  Is  Illustrated 
In  Figure  3.*  A  default  value  may  be  provided  If  and  only  If  a  location  for  a 
value  Is  present. 


Insert  Figure  3  about  here 


In  the  experiment,  we  assume  that  practice  Is  a  motoric  and  visual 
stimulus,  and  a  narrated  film  is  a  visual  and  verbal  stimulus.  Each  (practice 
and  film)  can  create  locations  for  all  three  media  (motoric,  visual,  and 
verbal),  but  each  does  so  more  or  less  strongly.  For  example,  practice  creates 
locations  (and  possibly  default  values)  for  verbal  material,  but  the  values.  If 
they  are  defined,  are  more  Imperfect  than  If  the  verbal  material  were  actually 
presented. 

With  the  above  theoretical  background,  we  can  Interpret  the  experimental 
results.  We  first  note  that  the  task  the  subjects  performed  (building  from 
memory)  Is  highly  biased  toward  a  motoric  and  visual  stimulus.  To  perform  the 
task,  a  subject  must  have  an  executable  concept  that  Is  motoric  and  visual;  the 
subject  Is  not  tested  on  any  verbal  measure. 

At  zero  delay,  the  groups  line  up  as  follows:  MM  ■  MF  ■  FM  >  FF  ■  M  >  F. 

For  the  three  top  groups,  we  do  not  know  whether  subjects  formed  one  or  two 
concepts.  (We  hypothesize  about  this  only  on  performance  after  a  delay.)  But 
clearly,  a  repeated  motoric  element  dominates,  and  the  MF  and  FM  groups  perform 
equally  well  (see  Table  1).  The  MM  group  Is  missing  some  of  the  visual  and 
verbal  elements  provided  by  the  film,  but  with  two  practices,  default  values 
seem  to  be  good  enough. 

The  FF  and  M  groups  are  tied  for  second  at  zero  delay.  We  assume  that  the 
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concepts  formed  In  the  two  groups  are  similar.  Thus  some  default  values  must  be 
provided  In  each  case. 

The  F  group  Is  poorest.  Getting  motoric  values  from  the  film  Is  difficult, 
(see  Footnote  1.) 

After  a  week  delay,  the  groups  line  up  as  follows:  MF>MM»FM«M«FF> 
F. 

The  F  group  Is  still  poorest.  The  FF  and  M  groups  remain  together,  with  a 
change  of  order  from  their  zero  delay  ranking.  (FF  Is  slightly  but  not 
significantly  worse.)  We  note  that  the  film  contains  linguistic  elements,  which 
are  lost  after  a  week  (Baggett  A  Ehrenfeucht,  1983).  This  loss  may  cause  a 
greater  deterioration  in  performance  for  FF. 

The  performance  of  the  MM  group  Is  not  statistically  different  from  that  of 
the  M  group  after  a  week  (although  they  were  significantly  different  at  zero 
delay).  This  means  that  the  decay  of  the  reinforced  concept  (MM)  occurs  at  a 
faster  rate  than  the  decay  of  the  same  unreinforced  concept  (M). 

Performance  of  the  FM  group  after  a  week  Is  statistically  the  same  as  that 
of  the  M  group  and  significantly  worse  than  the  MF  group.  This  Is  theoretically 
interesting.  We  think  that  the  FM  group  created  two  separate  concepts.  The 
first  has  audiovisual  components  with  undefined  values  that  are  motoric.  The 
second  has  visual  and  motoric  components  with  undefined  verbal  values.  The 
values  from  the  second  do  not  seem  to  be  transferred  to  the  first;  the  two 
concepts  do  not  become  Integrated.  Possibly  In  practicing  second,  the  subject 
has  the  motoric  elements  needed  for  the  task  and  Is  not  motivated  to  look  back 
to  the  undefined  values  In  his  or  her  audiovisual  concept.  The  subject  has 
everything  needed,  namely,  the  subject  has  Just  successfully  built  the 
helicopter. 

We  hypothesize  that.  In  building  from  memory,  subjects  In  the  FM  group  used 
only  one  of  their  concepts,  the  one  formed  from  building  from  the  model 
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(containing  visual  and  motoric  elements).  Thus,  their  performance  equals  that 
of  the  M  group. 

The  MF  group  performed  best  (and  significantly  so)  after  a  delay.  We 
hypothesize  that  they  created  a  single  unified  concept  from  their  stimulus 
presentations.  In  building  first  from  a  model,  they  built  motoric  components 
with  defined  values.  In  watching  the  film,  they  formed  an  audiovisual  concept 
with  undefined  motoric  values.  The  subjects  seemed  to  connect  the  two: 
audiovisual  back  to  motoric;  they  filled  the  slots  in  the  audiovisual  concept 
with  their  practiced  motoric  values,  forming  one  Integrated  concept.  The 
concept  contained  all  elements  (motoric,  visual,  and  verbal),  and  a  subject 
simply  executed  It  In  performing  the  task. 

Our  main  result  Is  the  superior  performance  of  the  MF  group  after  a  week. 
We  propose,  based  on  this  result,  that  elements  In  people's  concepts  form  a 
precedence:  primary  Is  motoric,  second  Is  visual,  and  third  is  language 
(symbolic  Information).  To  form  lasting  concepts,  the  order  In  which  the 
multimedia  Information  enters  a  concept  seems  Important.  From  this  experiment, 
motoric  Information  should  enter  first,  and  audiovisual  second. 

In  another  study  (Baggett,  In  press,  b)  the  soundtrack  of  a  film  was 
shifted,  so  that  an  object  shown  In  the  visual  image  and  Its  corresponding 
verbal  label  could  be  In  synchrony  or  the  visual  image  could  precede  or  follow 
the  spoken  label.  The  best  recall  of  a  name,  given  the  object,  was  for  the 
groups  with  visuals  and  verbals  In  synchrony  and  with  visuals  preceding  verbals 
by  up  to  seven  seconds.  The  principle  arising  from  that  study  was  that  for  good 
visual /verbal  associations  to  be  made,  one  should  show  and  tell  In  synchrony,  or 
show  first  and  tell  second,  and  one  should  not  tell  first  and  show  second.  (A 
similar  result  was  obtained  by  Baggett  A  Ehrenfeucht,  1983.)  Thus,  In  the 
framework  given  here,  visual  elements  should  arrive  at  a  concept  ahead  of,  or  In 
synchrony  with,  their  verbal  labels.  In  order  for  Integrated  concepts  to  be 


Page  19 


formed. 

The  theoretical  explanation  given  here  Is  ad  hoc.  But  the  fact  that  It  is 
consistent  with  previous  results  gives  It  credibility.  We  think  that  It  Is  a 
viable  hypothesis  that  we  can  expect  to  find  In  a  broad  range  of  cognitive 
research. 

We  note  that  the  correlations  between  structural  scores  and  ratings  given 
on  the  questionnaire  about  how  much  one  had  played  previously  with  assembly  kits 
(on  a  scale  of  zero  to  three)  were  positive  In  all  24  (6x2x2)  cases.  The 
correlations  ranged  from  .05  to  .60  and  averaged  .28.  The  correlations  mean 
that  previous  experience  on  similar  tasks  and  performance  on  the  task  performed 
here  are  related.  The  mean  ratings  of  previous  experience  ranged  from  1.30  to 
1.67  In  each  of  the  twelve  groups  (combining  males  and  females).  It  Is 
Interesting  to  note  that  the  average  rating  for  females  Is  1.31  and  for  males  is 
1.66.  The  ratio  of  these  two  Is  78.9%,  which  Is  approximately  the  ratio  of  the 
structural  scores  for  females  to  those  for  males  (46.23/61.34  ■  75.4%).  One 
reasonable  explanation  for  the  sex  difference  In  this  study  1$  simply  the 
difference  In  previous  experience  on  similar*  tasks. 

The  experimental  results  found  In  this  study  do  not  allow  us  to  recommend  a 
general  principle  for  multimedia  training  of  a  procedure.  Individual 
differences  In  performance  within  a  group  were  very  great.  For  example,  scores 
could  range  from  0  to  58,  and  an  actual  range  In  a  single  group  of  2  to  56  was 
common.  The  average  standard  deviation  In  a  group  was  over  20%. 

We  conclude  that  the  right  training  sequence  for  a  procedure  that  is  to  be 
performed  from  memory  varies,  depending  on  the  Individual.  And  this  brings  up 
the  question  of  Individualized  Instruction.  A  goal  of  our  future  research  Is  to 
discover  what  Individualized  Instruction  should  contain.  Specifically,  should 
Instruction  be  individualized  simply  by  varying  the  amount  given  to  different 
people,  depending  on  their  experience  or  skill?  Or  should  It  be  Individualized 
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by  giving  different  Modalities,  or  modalities  In  different  orders,  or  different 
conceptualizations  (divisions  Into  subassemblies),  etc?  A  second  goal  of  our 
future  work  Is  to  develop  a  small  number  of  brief  tests  that  can  be  easily  given 
to  subjects.  Performance  on  these  tests  would  be  used  to  (a)  predict 
performance  as  a  function  of  Instructions;  and  (b)  assign  a  person  to  an 
appropriate  Instructional  sequence. 

Until  such  tests  are  available,  we  recommend  that  a  person's  performance  be 
tested  after  practice,  after  film  Instruction,  and  after  various  amounts  and 
combinations,  to  see  which  gives  optimum  results.  If  such  testing  Is  not 
possible*,  the  Instructional  sequence  should  be  practice  first  and  film  second. 

Final  Remarks 

This  paper  has  Investigated  performance  on  an  assembly  task  as  a  function 
of  the  multimedia  Instructions  given.  It  has  put  forth  a  new  dependent  measure 
for  assembly  which  assesses  the  similarity  In  structure  of  an  object  built  from 
memory  and  a  correctly  built  object.  It  has  also  sketched  a  theoretical 
framework  based  on  multimedia  concept  formation  which  Is  helpful  In  Interpreting 
the  empirical  findings.  We  hope  that  the  new  methodology,  the  framework,  and 
the  practical  results  will  be  useful  In  a  variety  of  situations. 
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Footnotes 


This  research  was  supported  by  the  Office  of  Naval  Research  Contract 
IN00014-78-C-0433,  NR  157-422.  Some  of  the  results  were  reported  at  the  23rd 
Meeting  of  the  Psychonomlcs  Society  In  Minneapolis,  November  1982.  Thanks  go  to 
Jerl  Bacon,  Glnl  Kamanl,  Cynthia  Russell,  Joseph  Smith,  and  Lynn  Walllck  for 
helping  to  collect  and  score  the  data. 

*In  actuality,  we  think  the  situation  Is  slightly  different.  First,  a 
stimulus  In  one  modality  creates  a  location  (with  some  probability)  for  the  next 
lower  modality  (motoric  creates  a  visual  location;  visual  creates  a  verbal 
location;  see  Figure  3.)  Location  creation  proceeds  easily  In  the  downward 
direction.  Suppose  a  motoric  stimulus  creates  a  visual  location.  If  the  visual 
location  is  given  a  value  by  default,  this  causes  a  verbal  location  to  be 
created.  Thus,  a  motoric  stimulus  can  (Indirectly)  cause  a  location  two  steps 
away  to  be  created. 

If  a  visual  stimulus  is  presented,  a  location  Is  created  for  'ts  veob#‘> 
label.  The  location  Is  not  necessarily  filled  with  a  value. 

Location  creation  In  the  upward  direction  (verbal  to  visual,  or  visual  to 
motoric)  occurs  with  a  lower  probability.  It  Is  not  very  likely  that  a  verbal 
stimulus  can  cause  motoric  locations  to  be  formed.  One  does  not  directly  encode 
actions  from  words. 

This  report  Is  technical  report  #125  of  the  Institute  of  Cognitive  Science's 
technical  report  series. 


Table  1:  Percentage  of  Correct  Connections  in 
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Figure  Captions 


Figure  1.  The  model  helicopter,  built  from  54  pieces  of  the  Fischer  Technlk  50 
assembly  kit. 

Figure  2.  An  abstract  graph  of  the  model  helicopter  shown  In  Figure  1.  The 
nodes  represent  pieces  and  the  links  represent  physical  connections.  The  nooes 
are  labeled  1  to  54,  to  correspond  to  specific  pieces  In  the  model.  For 
example,  nodes  1  and  6  correspond  to  the  blades,  and  nodes  45  and  53  correspond 
to  the  tires.  For  scoring  performance,  the  graph  of  a  helicopter  built  from 
memory  was  compared  to  the  graph  of  the  correctly  built  one  (shown  above),  and 
the  number  of  correct  connections  was  counted. 

Figure  2* 

Note:  Squares  Indicate  locations.  Horizontal  bars  indicate  defined  values.  A 
question  mark  Indicates  that  the  value  Is  either  undefined  or  defined  by 
default. 

In  the  theoretical  framework,  motoric  elements  (A  above)  create  concepts  with 
defined  motoric  values,  and  with  locations  for  visual  (and  possibly  linguistic; 
see  Footnote  1)  elements.  These  locations  have  values  that  are  defined  by 
default  or  left  undefined.  Visual  elements  (8  aoove)  create  concepts  with 
defined  visual  values,  and  with  locations  for  motoric  and  linguistic  elements. 
These  locations  have  values  that  are  defined  by  default  or  left  undefined. 
Linguistic  elements  (C  above)  create  concepts  with  defined  linguistic  values, 
and  with  locations  for  visual  elements.  These  locations  have  values  that  are 
either  defined  by  default  or  left  undefined.  (See  also  Footnote  1.) 


Appendix 

The  film  contains  856  words  of  narration.  It  begins  as  follows: 

The  helicopter  consists  of  54  pieces  from  an  assembly  kit.  As  the 
helicopter  Is  pushed  along  a  surface.  Its  two  gears  mesh  and  Its  blades  turn. 
The  purpose  of  this  film  Is  to  show  you  step  by  step  how  to  build  the 
helicopter.  It  consists  of  seven  sections. 

The  tall  assembly  connects  to  the  lower  body.  The  propeller  assembly 
attaches  to  the  upper  body.  The  C-clIp  slides  on  the  rod,  and  the  main  gear 
attaches  to  the  rod.  The  lower  body  fits  on  the  rod . 
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